Goto

Collaborating Authors

 Jennings County


Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment

Liu, Yang, Yao, Yuanshun, Ton, Jean-Francois, Zhang, Xiaoying, Guo, Ruocheng, Cheng, Hao, Klochkov, Yegor, Taufiq, Muhammad Faaiz, Li, Hang

arXiv.org Artificial Intelligence

Ensuring alignment, which refers to making models behave in accordance with human intentions [1,2], has become a critical task before deploying large language models (LLMs) in real-world applications. For instance, OpenAI devoted six months to iteratively aligning GPT-4 before its release [3]. However, a major challenge faced by practitioners is the lack of clear guidance on evaluating whether LLM outputs align with social norms, values, and regulations. This obstacle hinders systematic iteration and deployment of LLMs. To address this issue, this paper presents a comprehensive survey of key dimensions that are crucial to consider when assessing LLM trustworthiness. The survey covers seven major categories of LLM trustworthiness: reliability, safety, fairness, resistance to misuse, explainability and reasoning, adherence to social norms, and robustness. Each major category is further divided into several sub-categories, resulting in a total of 29 sub-categories. Additionally, a subset of 8 sub-categories is selected for further investigation, where corresponding measurement studies are designed and conducted on several widely-used LLMs. The measurement results indicate that, in general, more aligned models tend to perform better in terms of overall trustworthiness. However, the effectiveness of alignment varies across the different trustworthiness categories considered. This highlights the importance of conducting more fine-grained analyses, testing, and making continuous improvements on LLM alignment. By shedding light on these key dimensions of LLM trustworthiness, this paper aims to provide valuable insights and guidance to practitioners in the field. Understanding and addressing these concerns will be crucial in achieving reliable and ethically sound deployment of LLMs in various applications.


Report: Hardware vendors will reap the rewards of AI's move to the edge

#artificialintelligence

Artificial intelligence is poised to take a large step out of the cloud and into edge computing, which will benefit edge AI hardware vendors. The shift from the cloud to edge AI, which includes devices, gateways and on premise services, will be driven by machine learning, or inference, and then by training, according to a report by ABI Research. Edge AI inference will increase from just 6% last year to 43% by 2023, according to ABI. When paired with machine learning and analytics, AI holds promise for Internet of Things applications, virtual reality, autonomous vehicles, and the rollout of 5G services. While there's still a lot of ground to cover before autonomous vehicles and virtual reality reach mass deployments, the report highlighted how AI can push adoption rates forward across various verticals.


Evolution of E-commerce: The possibilities of tomorrow

#artificialintelligence

The rapid growth of e-commerce is driving deep changes in logistics, from tightening up trucking capacity to elevating the importance of final-mile delivery processes. To respond, logistics managers now need to think in terms of systems that they can leverage today to make processes more efficient, while also keeping an eye on longer-term developments that will reshape tomorrow's possibilities. To gain a sharper picture of what e-commerce-related technologies logistics professionals should be watching, we reached out to consultants and analysts to develop a short-list of trends and solutions that need to be top of mind for today's savvy logistics professional. Several of thee include solutions currently in use, such as predictive analytics, supply chain control towers, and the continued digitization of freight forwarding; however, many, including blockchain-based traceability, driverless trucks, and even the advent of hyperloops, are all working through development, but promise to present bright new options in the future. Companies have long had dashboards to organize their metrics, but until recently, most of these approaches have been descriptive, meaning they look at past or current trends.